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ABSTRACT 

To determine the appropriateness of existing writing 
assessment instruments, to set directions for future assessment, and 
to ensure that assessment instruments are consistent with the 
requirements of the new English curriculum of the Northern Territory 
(Australia), teachers were sampled to determine how they used the 
booklets of moderated writing samples in 1991 and to gain other 
information about their regard for the moderated writing samples. A 
survey questionnaire was sent to teachers in years 5 and 7 in 71 
schools and to various administration officials. A total of 86 
teachers responded. Teachers believed, as they had in a 1989 survey, 
that the booklets containing the moderated samples and the framework 
of descriptors provided for assessment are useful resources for 
assessment of student writing. Nevertheless, many teachers did not 
know of or did not have access to the booklets. Year-7 teachers made 
more use of the booklets than did Year-5 teachers. Informal 
discussions made it clear that teachers are interested in having 
access to collections of moderated samples of early childhood student 
writing. With full implementation of the new curriculum and annual 
systero-wide moderation of student writing at years 5 and 7, use of 
the moderated samples will probably increase. The booklet survey is 
included. (SLD) 
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BACKGROUND 



When the Primary Assessment Program began, three types of assessment 
procedures in Reading and Writing were put in place. The method that was 
agreed to for the assessment of students' writing at Years 5 and 7 was that of 
'moderation'. This meant that assessment instruments were to be developed for 
teachers to use to assess their stiidents' written expression. 

In 1983, 1987 and 1989, samples of writing produced by Northern Territory urban 
students at Years 5 and 7 were assessed by teachers According to 'frameworks of 
descriptors' for various genres and then submitted for system-wide moderation 
by a panel of school and office-based teachers. 

Booklets containing the annotated samples, classified in bands of Low, 
Satisfactory and High levels of competency, were produced for use by teachers as 
'benchmark' assessment instruments. The genres covered were: 

• 1983 Narrative (literary gerire) 

• 1987 Narrative (literary geru-e) and report (non-hterary genre) 

• 1989 Argument and Explanation (non-literary genres). 

In 1991, a 'framework of descriptors' for assessing narrative poetry in ballad form 
was developed and included in an information booklet which was circulated to 
all urban primary schools. Teachers assessed the poems produced by their 
students and submitted them for system-wide moderation. A selection of these 
samples was included in the booklet which will become available for teachers to 
use as an assessment instrument beginning in 1992. 

With the exception of an evaluation of the Primary Assessment Program 
namely. The Primary Assessment Program: A Descriptive Study Incorporating 
Teacher Perceptions (Richards, 1989), the extent of the use made of the booklets 
over the years has been unmeasured and therefore largely unknown. 

In 1990, the NT Board of Studies endorsed the Primary Assessment Committee's 
recommendation that some form of system-wide moderation of students' 
writing take place armually from 1991. 

As tiiis is now current policy, it means that longer term strategic planning for 
writing assessment at Ye-^rs 5 and 7 is necessary. It was considered essential that 
teachers be given an opportunity to have input into this process. With the 
mandatory introduction of the new T-10 English curriculum in 1992, it is timely 
that past practice be considered and future direction set. 



PURPOSE OF THE SURVEY 

In order to determine the appropriateness of existing writing assessment 
instruments, to set direction for the development of future instruments that 
continue to meet the needs of teachers, and to ensure that the 'instruments' are 
consistent with requirements of the new T-10 English curriculum, there was a 
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need to consult with all teachers of Years 5 and 7 in urban primary schools. This 
consultation process sought to: 

• determine the use teachers made of each booklet of moderated samples of 
student writing during 1991 

• quantify the extent to which teachers used (in 1991) the 'frameworks of 
descriptors* in each of the genres associated with the moderated writing 
samples 

« 

• gain information about teachers' assessments of the usefulness of the 
booklets of sample scripts and the 'frameworks of descriptors' 

• find out what additional genres (if any) teachers would wish to see 
moderated for the purposes of constructing more booklets of samples 

• gain from teachers any additional comments and suggestions that would 
enhance the formal writing assessment process. 

AIMS OF THE SURVEY 

The data and information that was gathered from schools will be used to: 

• assess whether or not the existing booklets of moderated samples of 
writing and the 'frameworks of descriptors' are still considered by teachers 
to be relevant and useful 

• set direction for future system-wide writing assessment at Years 5 and 7. 
LIMITATIONS 

Constraints imposed by limited time and human resources did not allow 
interviews to be conducted with individual teachers, therefore the survey 
questiormaire was the only method of data collection. 

As the size of the Year 5/7 teacher population was not determined in this survey, 
it was not possible to quantify the precise proportion of the population accounted 
for by the responses received. 

SURVEY METHODOLOGY 

A survey questionnaire (see Appendix) was developed and field trialled on a 
sample of six teachers in schools and six officers of the Curriculum and 
Assessment Division. Adjustments were made to the questionnaire form on the 
basis of comments received from the trial 'respondents'. The shortness of the 
questionnaire was deliberate in order to keep 'respondent burden' to a 
minimum. 
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Questionnaire forms were sent to all teachers of Years 5 and 7 in NT government 
and non-government urban primary schools (71 schools); office-based English 
advisory staff in both northern and southern regions; all regional 
superintendents and teachers at the Rapid Creek and rural (Humpty Doo) 
Language Centres (11 office-based and 'other establishments' staff). No names of 
respondents were required for the survey. 

The number of forms sent to schools was based on a proportion of the total staff 
numbers for each. 

Provision was made on the form for those teachers with multiple year level 
classes or 'family' groups including students in both Years 5 and 7 levels. Where 
teachers completed the form for both year levels, the response was counted as 
two separate responses. 

Data recorded on the forms was entered into a database for storage and analysis. 

A total of 86 teachers including 41 Year 5 and 40 Year 7 teachers from 25 
government and non-government schools responded to the survey. Three 
respondents from the Year 5 group, and 2 from the Year 7 group supplied open- 
ended comments only. An additional 6 partially completed forms were received 
but no year level was identifiable for these. Given that the size of the suiTey 
population was not known, a best estimate of the return rate of responses was 
calculated as being approximately 40-45 per cent. 

All percentages appearing in this paper are rounded. 

DISCUSSION 

The two main features of the booklets of moderated samples are the 'frameworks 
of descriptors' for each genre, and the graded annotated samples of student 
writing. Both of these can be used in tandem, and constitute the assessment 
'instrument'. It was essential therefore to canvass teacher opinion regarding 
their use of both the 'frameworks of descriptors' and the annotated writing 
samples. 

Non-users of the frameworks and the samples in the Year 5 and Year 7 groups 
were different individuals and did not represent any constant core of 
respondents. 

Use Made of Frameworks of Descriptors in 1991 

Table 1 shows the amount of use made of the 'frameworks of descriptors' by the 
respondent group of teachers in 1991. The data presented in Table 1 and in 
Figure 1, show that the Year 5 teacher respondent group (38 out of 41 responded 
to this section of the questionnaire) made considerably more use of the narrative 
and report 'frameworks of descriptors' than they did of the argument and 
explanation frameworks. However, Year 7 teachers' use of the frameworks (38 
out of 40 responded to this section of the questionnaire) was fairly consistent 
across all four genre. 



Table 1: Usage of Frameworks of Descriptors in 1991 by Genre and Year 
Level 



Genre Yr5 Yr5 Yr7 Yr7 

Teachers Total Teachers Total 

Response Response 



Narrative "6 (95%) 38 30 (79%) 38 

Report 32 (84%) 38 33 (87%) 38 

Argument 17 (45%) 38 29 (76%) 38 

Explanation 20 (53%) 38 30 (79%) 38 



Figure 1: Usage of Frameworks of Descriptors in 1991 by Geitre and Year 517 

Level 
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Narrative Report Argument Explanation 



Degrees of Usefitlness of the Frameworks of Descriptors 

The survey questionnaire invited teachers to indicate on a three-point scale 
(not useful / useful / very useful), the degree to which the frameworks had been 
of use to them in assessing their students* writing in the four genre. Thirty-eight 
teachers ^t each of Years 5 and 7 responded to this section of the questionnaire. 

Table 2 and Figures 2 and 3 desaibe the response from teachers at Years 5 and 7. 
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Table 2: Usefulness of Framewoiks of Deiscriptors by Genre and Year 5/7 
Level 

Genre Not Useful Very Not Useful Very 

useful useful useful useful 

(Year 5) (Year 5) (Year 5) (Year 7) (Year 7) (Year 7) 



Narrative 1 19 14 0 14 15 

Report 1 12 18 0 15 17 

AirgunieiU 1 1 13 0 12 17 

Explanation 1 9 11 0 11 17 



Information depicted on Table 2 and Figure 2 shows that only one teacher from 
the Year 5 respondent group indicated that the 'frameworks of descriptors' were 
not useful The entire Year 7 teacher respondent group (see Figure 3) believed 
that the frameworks were either useful or very useful to them. 

In terms of overall numbers, the narrative and report frameworks were rated as 
being the most useful by the Year 5 teacher group (see Figure 2). This is 
consistent with the level of use made of the frameworks and writing samples in 
those two genres. 

Although fewer Year l> teachers reported that thc-.y considered the argument and 
explanation frameworks useful, and given the smaller numbers that actually 
used them, proportionately the usefulness of the argument 'framework of 
descriptors' was rated highly by those who used it. 

Interestingly, the report and argument frameworks were put consistently into 
the very useful category while the majority of Year 5 teachers indicated that the 
narrative and explanation frameworks were in the useful category. 



Figuw,' 2: Usfcfulness of Frameworks of Descriptors by Genre and Year 5 Level 




Narrative Report Aigument Explanation 



Genre 



5 



11 



Flgiire 3: Usefulness of Frameworks of Des^ptors by Genre and Year 7 Level 




M not useful 
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Figure 3 demonstrates that the entire Year 7 teacher respondent group believed 
that all four framev^orks v/ere either useful or r;ery useful The majority of this 
group indicated that the four frameworks were very useful rather than useful 
only. While there was little variation in the teachers' ratings of the usefulness of 
all four 'frameworks of descriptors', the explanation framework was the most 
highly rated in usefulness. 

Overall numbers of responses showed that generally, tl\e report framework was 
of the most use to the Year 7 teacher group. The trend of fairly evenly distributed 
usage of the frameworks in all four genres was reflected in the teachers' 
perceived usefulness of them. 

Use Made of Writing Samples in 1991 

Teachers were invited to indicate on the questionnaire, if they had used (in 1991) 
the moderated writing samples of writing in the four genre. Table 3 and Figure 4 
summarise the responses by the 38 Year 5 and 38 Year 7 teachers who responded 
to this section of the questiomiaire. 

Table 3: Usage of Writing Samples in 1991 by Genre and Year Level 



Genre 


Yr5 

Teachers 


Yr5 

Total 

Response 


Yr7 

Teachers 


Yr7 

Total 

Response 


Narrative 


30 (79%) 


38 


29 (76%) 


38 


Report 


30 (79%) 


38 


28 (74%) 


38 


Argument 


17 (45%) 


38 


27 (71%) 


38 


Explanation 


19 (50%) 


38 


29 (76%) 


38 
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Seventy-nine per cent of the Year 5 teacher group used narrative samples and 79 
per cent used the report samples. This was in contrast with the 45 per cent that 
used the argument samples and the 50 per cent that reported using the 
explanation ones. This pattern was consistent with their usage of the 
'frameworks of descriptors' (see Figure 1). 

Usage of samples in the Year 7 respondent group showed that 76 per cent used 
narrative, 74 per cent used report, 71 per cent used argument and 76 per cent used 
explanation samples. Figure 4 shows that there was very little variation in 
Year 7 teachers' use of the writing samples in all four gem-es. 

Year 7 teachers' pattern of usage of both the frameworks and the writing samples 
(see Figure 1) was highly consistent This same degree of consistency between 
use of frameworks and samples was present for the Year 5 teacher group also. 



Figure 4: Usage of Writing Samples in 1991 by Genre and Year 5/7 Level 
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Table 4: Usefulness of Writing Samples by Genre and Year 5/7 
Level 



Genre Not Useful Very Not Useful Very 

useful useful useful useful 

(Years) (Year 5) (Year 5) (Year 7) (Year 7) (Year 7) 



Narrative 1* 17 12 0 12 17 

Report 1* 13 14 0 14 15 

Argument 1* 7 10 0 12 16 

Explanation 1* 10 9 0 11 13 



*The one teacher who did not believe the writing samples to be of use was the same respondent who 
expressed this belief about the usefulness of the Year 5 'frameworks of descriptors*. 

ERLC ^ 



Reference to Figure 5 reveals that Year 5 teachers consistently rated writing 
samples in the narrative and report genres as of the n\ost overall use. Samples 
in the narrative genre were the most used by Year 5 teachers but were rated as 
useful only. Samples in the report, argument and explanation genres were less 
used than those in the narrative genre but were regarded by those who used 
them as being very useful. 



Figure 5: Usefulness of Writing Samples by Geiue and Year 5 Level 
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The entire Year 7 group of teachers indicated that the writing samples for all four 
genre were either useful or very useful More Year 7 teachers regarded the 
samples as being very useful rather than just useful (see Table 4 and Figure 6). 



Figure 6: Usefulness of Writing Samples by Genre and Year 7 Level 
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Genres for Future Booklets of Moderated Samples of Students* Writing 



The questionnaire invited teachers to nominate the genres of student writing 
samples they would wish to see modereted in the future, for the purpose of 
compiling them into booklets similar to those already in use. 

Of the 86 returns, 34 recorded a response to this question even though some of 
the suggestions included requests for booklets of exemplar pieces of v^iting that 
are not generally regarded as 'genres'. Some of these included letter writing, 
responses to literature, diaries, book reviews and drama /script writing. This 
accounted for approximately 29 per cent of the group that responded to this 
question. 

Eighteen per cent of the respondent group indicated a preference for having 
access to a booklet of moderated samples in the procedure genre. Twelve per 
cent suggested that it would be useful to have moderated samples in the recount 
genre, while six per cent of the respondent group believed that the genres of 
writing already moderated were 'adequate*. Approximately nine per cent of this 
group believed that booklets of student writing samples should be compiled in 
*air genres. 

Twenty-one p - cent of the respondents welcomed the opportunity to have 
access to a booklet of moderated narrative poetry in 1992. 

General Comments 

The two most notable features of teacher response to the 'General Comments' 
section of the questionnaire were: 

• the considerable number of teachers who were not aware of the existence 
of the booklets of writing samples until receiving the survey questionnaire 

• the number of teachers who commended the booklets and declared them 
to be a useful resource. 

With regard to the efficacy of the booklets, one teacher's comments were that 
they were 'extremely useful in staff development exercises, as well as for 
classroom practice, planning, implementation and evaluation'. 

It appeared that a number of teachers were not only using the booklets as 
assessment instruments but were using them as teaching resources as well. 
Some teachers indicated that they had made copies of the 'high' competency 
samples available to their students to use as models of good writing in particular 
genres. One respondent suggested that more examples of 'high' competency 
samples be included in the booklets. 



Examples of other comments offered were: 



• that one draft of each moderated sample be included in the booklets to 
illustrate development of each of the final scripts 

• that there should be one copy of each booklet made available to every 
teacher of Years 5 and 7 

• that some additional samples of writing in the argument and explanation 
genres would be useful. 

The questionnaire gave teachers an opportimity to elaborate on any aspect of the 
existing booklets that they considered to be of no use. Two of the 86 respondents 
made use of this section. 

One respondent reported that he/she did not make use of samples rated as 
'poor'. The other believed that the annotations accompanying the report on 
'Spider' (Children's Writing in the Northern Territory: Moderated Examples of 
Reports and Narratives from Students in Years 5 and 7, p.ll ), were of limited 
value because they did not adequately explain why the sample was rated as 
'poor'. 

Conciusions 

The evaluation carried out by Richards in 1989 reported that teachers of Years 5 
and 7 in urban schools believed the booklets then in existence were 'good 
assessment support material' and therefore useful to ihem. From data and 
information gathered from the 1991 survey, it was clear that teachers still 
believed the booklets containing the moderated samples of writing and the 
'frameworks of descriptors' to be a useful resource for teaching and for 
assessment of student writing. 

It was obvious from the responses that many tea(^hers, for one reason or another, 
had not had access to personal copies of the booklets, or did not know of their 
existence. This situation is of some concern, and it is obvious that strategies need 
to be developed to help address the issue. 

Another notable feature that emerged from the data was that of the use teachers 
made of 'frameworks of descriptors' and writing samples in the argument and 
explanation genres. The data showed that Year 7 teachers made more use of 
these, comparatively, than did Year 5 teachers. However, those Year 5 teachers 
who did use them, rated their overall usefulness highly. The Year 5 teachers 
who indicated that they did not use the argument and explanation materials, did 
not record in the space provided on the survey form that they believed these 
materials to be of no use. The conclusions that may be drawn from this are that: 

• Year 5 teachers worked less with their students in the argument and 
explanation genres than did Year 7 teachers 



• a significant group of Year 5 teachers believed the materials in the two 
geiu-es to be of no use but did not state this in the space provided on the 
questionnaire. 

Given the number of teachers who indicated a desire to have access to moderated 
samples of student writing in a variety of other literary and non-literary forms, 
consideration could be given to this (subject to available resources) in future. 

Interest was shown by some respondents to this survey in booklets of samples 
being produced for year levels other than those involved in the Primary 
Assessment Program. Some urban and rural primary schools have taken the 
initiative to compile collections of writing samples for all year levels, other than 
Years 5 and 7, within their own schools or regions. 

Informal discussions with teachers revealed a growing awareness of the 
importance they attach to having access to collections of moderated samples of 
early childhood student writing. While these discussions were outside the scope 
of this survey, the views expressed oy the teachers concerned are nonetheless 
valid. 

With the full implementation of the new T-10 English curriculum in urban 
schools in 1992, annual system-wide moderation of student writing at Years 5 
and 7, and the increased trend towards a genre-based approach to the teaching of 
writing, a reasonable prediction would be that teacher use of booklets of 
moderated samples of student writing in a range of genres will become 
increasingly widespread in Northern Territory schools. 
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